Fast ConvNets Using Group-wise Brain Damage

摘要

We revisit the idea of brain damage, i.e. the pruning of the coefficients ofa neural network, and suggest how brain damage can be modified and used tospeedup convolutional layers. The approach uses the fact that many efficientimplementations reduce generalized convolutions to matrix multiplications. Thesuggested brain damage process prunes the convolutional kernel tensor in agroup-wise fashion by adding group-sparsity regularization to the standardtraining process. After such group-wise pruning, convolutions can be reduced tomultiplications of thinned dense matrices, which leads to speedup. In thecomparison on AlexNet, the method achieves very competitive performance.